Image capturing apparatus and image capturing method

ABSTRACT

In a high speed image capturing state, a camera signal processing circuit is not needed to perform a signal process at a high screen rate, but at a regular screen rate. In the high speed image capturing mode, raw data of 240 fps received from an image sensor  101  are recorded on a recording device  111  through a conversion processing section  201  and a recording device controlling circuit  210 . Raw data that have been decimated and size-converted are supplied to a camera signal processing circuit  203  through a pre-processing circuit  202  and an image being captured is displayed on a display section  112  with a signal for which a camera process has been performed. In a reproducing state, raw data are read from the recording device  111  at a low screen rate according to a display performance of the display section  112  and the raw data that have been read are processed are processed by the pre-processing circuit  202  and the camera signal processing circuit  203  and a reproduced image is displayed by the display section  112.

TECHNICAL FIELD

The present invention relates to an image capturing apparatus and animage capturing method, in particular, to those that capture imagesusing a high speed image sensor.

BACKGROUND ART

In the past, high speed video cameras that can capture images at highspeeds have been known. For example, a video camera has accomplishedhigh speed image capturing by converting an image size per one image tobe processed at a high speed rate into 1/4 of the standard image sizeand placing these four images in an image of the regular rate (seePatent Document “Japanese Patent Application Laid-Open Publication No.HEI 8-88833”). Another video camera has accomplished high speed imagecapturing using a circuit structure that processes data received from asensor in parallel so as to increase a process amount per unit time (seePatent Document “Japanese Patent Application Laid-Open Publication No.HEI 8-251492”).

However, the high speed image capturing described in Patent Document“Japanese Patent Application Laid-Open Publication No. HEI 8-88833” orPatent Document “Japanese Patent Application Laid-Open Publication No.HEI 8-251492” was aimed to temporarily store a captured image in astorage device such as a VTR or a semiconductor memory, reproduce thecaptured image in slow motion, and analyze a very high speed motion andthe device itself had a complicated structure and was expensive. Thus,it was difficult to apply high speed image capturing systems asdescribed in Patent Document 1 or Patent Document 2 to portable imagecapturing devices that have been widespread as home-use devices,so-called camcorders (product names of devices in which a video cameraand a recorder are integrated in one unit), digital cameras, and soforth from view points of portability and power consumption.

With reference to FIG. 1, an image capturing apparatus designed takingaccount of such points will be described. The structure shown in FIG. 1has the same structure as that of an existing camcorder. In other words,an image capturing apparatus 100 shown in FIG. 1 includes an imagesensor 101, a pre-processing circuit 102, a camera signal processingcircuit 103, a conversion processing section 104, a compression anddecompression circuit 105, a memory control circuit 106, a memory 107, adisplay processing circuit 108, a compression and decompression circuit109, a recording device control circuit 110, a recording device 111, adisplay section 112, and a control section 113.

The image sensor 101 can select a high speed image capturing mode inwhich the image sensor 101 reads a signal at a first screen rate (alsoreferred to as frame rate) of 60 fps (fields/second) or more based onthe NTSC specifications or a regular image capturing mode in which theimage sensor 101 reads a signal at a regular second screen rate. Thescreen rate in the high speed image capturing mode is 240 fps that isfour times higher than that of the regular rate. The image sensor 101 isequipped with a CDS (Correlated Double Sampling) and an A/D converterand the image sensor 101 outputs captured image data.

The pre-processing circuit 102 performs an optically correcting processsuch as a shading correction for captured image data that are outputfrom the image sensor 101 and outputs a digital image signal. The camerasignal processing circuit 103 performs a camera signal process such as awhite balance adjustment process for the captured image data that arereceived from the pre-processing circuit 102.

The conversion processing section 104 performs a display decimation anda size adjustment to convert an image signal received from the camerasignal processing circuit 103 into an image signal having a screen rateand a screen size suitable for a display of the display section 112. Thedisplay decimation is performed only when an image signal received fromthe camera signal processing circuit 103 is output to the displayprocessing circuit 108. The display decimation decimates the number offields per unit time of the image signal captured by the image capturingapparatus 100 in the high speed image capturing mode to the number offields per unit time defined in the display standard of the displaydevice (60 fps in this case).

The compression and decompression circuit 105 performs acompression-encoding process for captured image data received from theconversion processing section 104 according to a still image encodingsystem, for example, JPEG (Joint Photographic Experts Group) or thelike. In addition, the compression and decompression circuit 105performs a decompression-decoding process for encoded data of a stillimage supplied from the memory control circuit 106. The memory controlcircuit 106 controls writing and reading image data to and from thememory 107. The memory 107 is a FIFO (First In First Out) type buffermemory that temporarily stores image data received from the memorycontrol circuit 106 and, for example, an SDRAM (Synchronous DynamicRandom Access Memory) or the like is used for the memory 107.

The display processing circuit 108 generates an image signal to bedisplayed on the display section 112 from an image signal received fromthe conversion processing section 104 or the compression anddecompression circuit 109, supplies the signal to the display section112, and causes it to display an image. The display section 112 iscomposed, for example, of an LCD (Liquid Crystal Display) and displays acamera-through image that is being captured or a reproduced image ofdata that have been recorded in the recording device 111.

The compression and decompression circuit 109 performs acompression-encoding process according to a moving image encodingsystem, for example, MPEG (Moving Picture Experts Group) or the like forimage data received from the conversion processing section 104. Inaddition, the compression and decompression circuit 109 performs adecompression-decoding process for encoded data of a moving imagesupplied from the recording device 111 and outputs the resultant data tothe display processing circuit 108. The display section 112 displays amoving image received from the display processing circuit 108.

The control section 113 is a microcomputer composed, for example, of aCPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (RandomAccess Memory), and so forth and totally controls each section of theimage capturing apparatus by executing programs stored in the ROM and soforth.

In the image capturing apparatus shown in FIG. 1, in the high speedimage capturing mode, a captured image signal of 240 fps received fromthe image sensor 101 is supplied to the camera signal processing circuit103 through the pre-processing circuit 102. The conversion processingsection 104 decimates an output image signal of the camera signalprocessing circuit 103 by 1/4 so that an output signal of 60 fps isobtained. The decimated and size-converted image signal is supplied tothe display processing circuit 108. The display processing circuit 108generates an image signal to be displayed on the display section 112,supplies the resultant image signal to the display section 112, andcauses it to display an image.

When receiving a record request of a high speed captured image from thecontrol section 113 according to the user's operation, the conversionprocessing section 104 sends an image signal of 240 fps to thecompression and decompression circuit 105. If necessary, the conversionprocessing section 104 reduces the size of the image signal receivedfrom the camera signal processing circuit 103 and sends the image signalto the compression and decompression circuit 105.

The compression and decompression circuit 105 compression-encodes theimage signal received from the conversion processing section 104according to the JPEG format. The memory control circuit 106 temporarilystores encoded data received from the compression and decompressioncircuit 105 into the memory 107. In such a manner, image data for apredetermined period are stored in the memory 107.

When receiving a read request for encoded data stored in the memory 107from the control section 113, the memory control circuit 106 reads theencoded data stored in the memory 107 at 60 fps and sends the encodeddata to the compression and decompression circuit 105. The compressionand decompression circuit 105 decompression-decodes the encoded datareceived from the memory control circuit 106 and sends the decoded datato the conversion processing section 104. When receiving a recordrequest for the recording device 111 from the control section 113, theconversion processing section 104 sends the image signal received fromthe compression and decompression circuit 105 to the compression anddecompression circuit 109. The compression and decompression circuit 109compresses the image signal received from the conversion processingsection 104 according to the MPEG format and stores thecompression-encoded signal to the recording device 111 through therecording device control circuit 110. The conversion processing section104 adjusts the size of the image signal of 60 fps received from thecompression and decompression circuit 105, sends the resultant imagesignal to the display processing circuit 108, and causes display section112 to display a reproduced image.

In the foregoing proposed image capturing apparatus shown in FIG. 1, inthe high speed image capturing mode of the image sensor 101, since thescreen rate of the output captured signal is high, the pre-processingcircuit 102 and the camera signal processing circuit 103 are required tooperate at a high speed. If the system is accomplished by an LSI or thelike, since the calculation scales of the pre-processing circuit 102 andthe camera signal processing circuit 103 are large in the whole system,a high speed process or a parallel process is not advantageous from theview point of circuit area and power consumption. Moreover, in thestructure shown in FIG. 1, since a captured image signal was temporarilystored in the memory 107, there was a problem that after the high speedimage capturing mode was stopped, it took a process time to decode datatemporarily stored in the memory 107 and to re-encode the data accordingto the regular record format.

DISCLOSURE OF THE INVENTION

Therefore, an object of the present invention is to provide an imagecapturing apparatus and an image capturing method that do not need tocause a camera signal processing circuit to perform a process at a highspeed screen rate and that are easy to operate, are produced at lowcost, and thereby have excellent portability.

To solve the foregoing problems, the present invention is an imagecapturing apparatus, comprising:

-   -   record medium control means for recording captured image data        having a first screen rate obtained from a solid state image        capturing device to a record medium and reading captured image        data having a second screen rate which is lower than the first        screen rate from the record medium;    -   conversion means for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   camera signal processing means for performing a camera signal        process for the captured image data having the second screen        rate obtained from the conversion means or the record medium        control means; and    -   display processing means for generating an image signal to be        displayed from an image signal obtained from the camera signal        processing means.

The present invention is an image capturing apparatus, comprising:

-   -   record medium control means for recording captured image data        having a first screen rate obtained from a solid state image        capturing device to a record medium and reading captured image        data having a second screen rate which is lower than the first        screen rate from the record medium;    -   conversion means for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   first camera signal processing means for performing a camera        signal process for the captured image data having the second        screen rate obtained from the conversion means;    -   second camera signal processing means for performing a camera        signal process for the captured image data having the second        screen rate obtained from the record medium control means; and    -   display processing means for generating an image signal to be        displayed from an image signal obtained from either the first        and second camera signal processing means,    -   wherein the first camera signal processing means is simply        structured in comparison with the second camera signal        processing means.

The present invention is an image capturing apparatus, comprising:

-   -   first record medium control means for recording captured image        data having a first screen rate obtained from a solid state        image capturing device to a first record medium and reading        captured image data having a second screen rate which is lower        than the first screen rate from the record medium;    -   conversion means for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   first camera signal processing means for performing a camera        signal process for the captured image data having the second        screen rate obtained from the conversion means;    -   second camera signal processing means for performing a camera        signal process for the captured image data having the second        screen rate obtained from the first record medium control means;    -   second record medium control means for recording an output image        signal of the second camera signal processing means to a second        record medium and reading an image signal from the record medium        at the second screen rate; and    -   display processing means for generating an image signal to be        displayed from either an output image signal obtained from the        first camera signal processing means or an output image signal        of the second record medium control means,    -   wherein the first camera signal processing means is simply        structured in comparison with the second camera signal        processing means.

The present invention is an image capturing method, comprising:

-   -   a record medium control step for recording captured image data        having a first screen rate obtained from a solid state image        capturing device to a record medium and reading captured image        data having a second screen rate which is lower than the first        screen rate from the record medium;    -   a conversion step for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   a camera signal processing step for performing a camera signal        process for the captured image data having the second screen        rate obtained from the conversion step or the record medium        control step; and    -   a display processing step for generating an image signal to be        displayed from an image signal obtained from the camera signal        processing step.

The present invention is an image capturing method, comprising:

-   -   a record medium control step for-recording captured image data        having a first screen rate obtained from a solid state image        capturing device to a record medium and reading captured image        data having a second screen rate which is lower than the first        screen rate from the record medium;    -   a conversion step for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   a first camera signal processing step for performing a camera        signal process for the captured image data having the second        screen rate obtained from the conversion step;    -   a second camera signal processing step for performing a camera        signal process for the captured image data having the second        screen rate obtained from the record medium control step; and    -   a display processing step for generating an image signal to be        displayed from an image signal obtained from either the first        and second camera signal processing step,    -   wherein the first camera signal processing step is simply        structured in comparison with the second camera signal        processing step.

The present invention is an image capturing method, comprising:

-   -   a first record medium control step for recording captured image        data having a first screen rate obtained from a solid state        image capturing device to a first record medium and reading        captured image data having a second screen rate which is lower        than the first screen rate from the record medium;    -   a conversion step for converting the captured image data having        the first screen rate obtained from the solid state image        capturing device into the captured image data having the second        screen rate;    -   a first camera signal processing step for performing a camera        signal process for the captured image data having the second        screen rate obtained from the conversion step;    -   a second camera signal processing step for performing a camera        signal process for the captured image data having the second        screen rate obtained from the first record medium control step;    -   a second record medium control step for recording an output        image signal of the second camera signal processing step to a        second record medium and reading an image signal from the record        medium at the second screen rate; and    -   display processing step for generating an image signal to be        displayed from either an output image signal obtained from the        first camera signal processing step or an output image signal of        the second record medium control step,    -   wherein the first camera signal processing step is simply        structured in comparison with the second camera signal        processing step.

In the image capturing apparatus according to the present invention,although a screen rate of an image capturing device is high, a camerasignal processing circuit is needed to always satisfy only a screen rateaccording to the display performance of a display section. Thus, it isnot necessary to perform a high frequency drive and a parallel processonly for high speed image capturing and thereby power consumption andcircuit scale can be reduced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an example of an image capturingapparatus that has been proposed;

FIG. 2 is a block diagram showing an image capturing apparatus accordingto a first embodiment of the present invention;

FIG. 3A and FIG. 3B are schematic diagrams used to describe an exampleof an image sensor according to the present invention;

FIG. 4 is a block diagram showing a more detailed structure of a part ofthe first embodiment of the present invention;

FIG. 5 is a data flow chart in a high speed image capturing modeaccording to the first embodiment of the present invention;

FIG. 6 is a data flow chart in an image reproducing state of the firstembodiment of the present invention;

FIG. 7 is a block diagram showing an image capturing apparatus accordingto a second embodiment of the present invention;

FIG. 8 is a data flow chart in a high speed image capturing mode of thesecond embodiment of the present invention;

FIG. 9 is a data flow chart in an image reproducing state of the secondembodiment of the present invention;

FIG. 10 is a block diagram showing an image capturing apparatusaccording to a third embodiment of the present invention;

FIG. 11 is a data flow chart in a high speed image capturing modeaccording to a third embodiment of the present invention;

FIG. 12 is a data flow chart in an image reproducing state of the thirdembodiment of the present invention;

FIG. 13 is a block diagram showing an image capturing apparatusaccording to a fourth embodiment of the present invention;

FIG. 14 is a data flow chart in a high speed image capturing mode of thefourth embodiment of the present invention; and

FIG. 15 is a data flow chart in an image reproducing state of the fourthembodiment of the present invention.

BEST MODES FOR CARRYING OUT THE INVENTION

Next, with reference to accompanying drawings, a first embodiment of thepresent invention will be described. As shown in FIG. 2, a capturedimage signal received from an image sensor 101 is supplied to an imagecapturing apparatus 200 according to the first embodiment of the presentinvention.

The image sensor 101 converts incident light of an object capturedthrough an optical system (including a lens, an infrared suppressionfilter, an optical low-pass filter, and so forth) into an electricsignal according to the photoelectric conversion. As the image sensor101, for example, a CMOS (Complementary Metal Oxide Semiconductor) typeimage capturing device is used. In the CMOS type image capturing device,photo diodes, line-column selection MOS transistors, signal wires, andso forth are two-dimensionally arranged to form a vertical scanningcircuit, a horizontal scanning circuit, a noise reduction circuit, atiming generation circuit, and so forth. As the image sensor 101, a CCD(Charge Coupled Device) that can capture images at high speeds may beused.

The image sensor 101 can be switched over between a high speed imagecapturing mode in which a signal is read at a first screen rate (alsoreferred to as the frame rate) higher than the regular screen rate (60fps (fields/sec) that is based on the specifications of the NTSC systemand a regular image capturing mode in which a signal is read at a secondscreen rate that is the regular screen rate. The screen rate of the highspeed image capturing mode is needed to be 240 fps that is four timeshigher than that of the regular rate. The image sensor 101 is internallyequipped with a CDS (Correlated Double Sampling), an A/D converter, andso forth and outputs a digitally captured image signal corresponding tothe matrix of pixels of the image sensor 101.

The image sensor 101 uses three image capturing devices that outputcaptured image signals, for example, of three-primary colors and obtainsone output line every four output lines of each image capturing deviceto accomplish a screen rate of 240 fps that is four times higher thanthat of the regular screen rate (60 fps). Assuming that the number ofpixels of one frame at the regular screen rate is, for example, 6.4million pixels, the number of pixels in the high speed image capturingmode is 1.6 million pixels.

FIG. 3A shows an example of an array of color filters according to thepresent invention. A square lattice array is inclined by 45° and each ofR and B filters is surrounded by a G filter. In this structure, whilenecessary and sufficient spatial frequency characteristics are obtainedfor R and B components on the human's visual sensitivity, a spatialfrequency characteristic of G component that is higher than that of eachof R and B components in the human's sensitivities can be improved incomparison with that of the conventional Bayer's array. The G componentbecomes a main component to generate a luminance signal. Thus, not onlythe resolution of luminance of an achromatic object, but that of achromatic object is improved and thereby the image quality is improved.

The color filter array shown in FIG. 3A is based on a method ofalternately reading pixels of two adjacent lines in one horizontalperiod at the regular screen rate as denoted by broken lines. In otherwords, in the regular image capturing mode, pixels are scanned and readin such a sequence.

On the other hand, at the high screen rate, horizontal scanning linesare decimated and read at a rate of one every four horizontal scanninglines. To deal with the high screen rate, as shown in FIG. 3B, assumingthat read outputs of six adjacent column pixels are denoted by CH1 toCH6, respectively, two A/D converters are disposed in common with threealternative columns (CH1, CH3, and CH5 and CH2, CH4, and CH6). The A/Dconverters convert, for example, one sample into digital data of 14bits. Instead, respective A/D converters may be disposed at columnpixels. CDSs (not shown) are disposed in the same manner as the A/Dconverters such that high speed reading can be performed. As the imagesensor 101, another structure in which three-primary color filters arearranged for one image capturing device may be used. Instead, imagecapturing devices using complementary color filters may be used.

The image capturing apparatus 200 includes a conversion processingsection 201, a pre-processing circuit 202, a camera signal processingcircuit 203, a display processing circuit 208, a recording devicecontrol circuit 210, a recording device 111, a display section, and acontrol section 213.

The conversion processing section 201 performs signal shunting anddisplay decimating for a digital image signal received from the imagesensor 101. The display decimating is performed only when a signal isoutput to the display processing circuit 208. The display decimating isa decimation of fields that satisfy the number of fields per unit timedefined in the display standard in the high speed image capturing modeof the image capturing apparatus 200 (in this case, 60 fps).

The pre-processing circuit 202 performs an optically correcting processsuch as a shading correction for a digital image signal that is outputfrom the image sensor 101 and outputs a resultant digital image signal.The camera signal processing circuit 203 performs a camera signalprocess such as a white balance adjustment process (also referred to asa development process, an image creation process, or the like) for theimage signal received from the pre-processing circuit 202. An outputsignal of the camera signal processing circuit 203 is supplied to thedisplay processing circuit 208.

The display processing circuit 208 generates an image signal to bedisplayed on the display section 112 from the image signal received fromthe camera signal processing circuit 203 and supplies the resultantsignal to the display section 112 to cause it to display an image. Thedisplay section 112 is composed, for example, of an LCD (Liquid CrystalDisplay) and displays a camera-through image that is being captured, areproduced image of data recorded on the recording device 111, and soforth. The display section 112 may be disposed outside the imagecapturing apparatus 200 and it may be provided with an interface for anexternal output instead of the display section 112.

The recording device control circuit 210 connected to the conversionprocessing section 201 controls writing and reading image data to andfrom the recording device 111. Data stored in the recording device 111are captured image data that have not been processed by the foregoingpre-processing circuit 202 and camera signal processing circuit 203 andare referred to as raw data in this specification.

As the recording device 111, a magnetic tape, a semiconductor memorysuch as a flash memory, a hard disk, or the like can be used. As therecording device 111, a non-attachable/detachable type is basicallyused. However, the recording device 111 may be attachable/detachablesuch that raw data can be retrieved to the outside. When raw data areretrieved to the outside, raw data that have been processed in thepre-processing circuit 202 are preferably retrieved. The camera signalprocess is performed, for example, according to software of an externalpersonal computer.

The control section 213 is a microcomputer composed, for example, of aCPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (RandomAccess Memory), and so forth and totally controls each section of theimage capturing apparatus by executing programs stored in the ROM and soforth.

FIG. 4 shows an example of a structure of the conversion processingsection 201, the pre-processing circuit 202, and the camera signalprocessing circuit 203. Captured image data received from the imagesensor 101 (including CDSs and A/D converters) are supplied to theconversion processing section 201. The conversion processing section 201is composed of switches SW1 and SW2, a decimation section 221, and asize adjustment section 222. The decimation section 221 performs adisplay decimation. The size adjustment section 222 changes the size ofan image to be displayed to an appropriate size. Either the decimationsection 221 or both of the decimation section 221 and the sizeadjustment section 222 decrease the screen rate of raw data in the highspeed image capturing state to the regular rate. The switches SW1 andSW2 are changed over in the recording state and the reproducing stateand a terminal selected in the recording state is denoted by r and aterminal selected in the reproducing state is denoted by p.

An output image signal of the size adjustment section 222 of theconversion processing section 201 is supplied to a shading correctioncircuit 231 of the pre-processing circuit 202. The shading correctioncircuit 231 corrects the brightness of the vicinity of the screen suchthat it does not become dark. An output signal of the shading correctioncircuit 231 is supplied to the camera signal processing circuit 203.

The camera signal processing circuit 203 is composed, for example, of asimultaneously forming circuit 241, a white balance correction section242, an aperture correction section 243, a gamma correction section 244,and a YC generation section 245 arranged in the order from the inputside. However, the structure of the camera signal processing circuit 203is not limited to that shown in FIG. 4. For example, the arrangementorder of these structural elements may be changed or a part of which maybe omitted.

The simultaneously forming circuit 241 interpolates lost pixels of eachcolor component. The simultaneously forming circuit 241 outputsthree-primary color signals (R, G, B) in parallel. Output signals of thesimultaneously forming circuit 241 are supplied to the white balancecorrection section 242. The white balance correction section 242corrects unbalancing of colors caused by a different color temperatureenvironment of an object and different sensitivities of color filters ofthe sensor.

An output of the white balance correction section 242 is supplied to theaperture correction section 243. The aperture correction section 243 isto perform a contour correction that extracts a portion where a signallargely changes and emphasizes the portion. An output signal of theaperture correction section 243 is supplied to the gamma correctionsection 244.

The gamma correction section 244 corrects input and outputcharacteristics such that the gradation is correctly reproduced when acaptured image signal is output to the display section 112. An outputsignal of the gamma correction section 244 is supplied to the YCgeneration section 245.

The YC generation section 245 generates a luminance signal (Y) and acolor difference signal (C). The luminance signal is generated bycombining the gamma-corrected RGB signals at a predetermined compositionratio. The color difference signal is generated by combining thegamma-corrected RGB signals at a predetermined composition ratio. Thegenerated luminance signal and color difference signal are supplied tothe display section 112 through the display processing circuit 208.

In the image capturing apparatus shown in FIG. 2, in the high speedimage capturing mode, as shown in FIG. 4, the control section 213controls the switches SW1 and SW2 of the conversion processing section201 to select the terminal r side. FIG. 5 shows signal flows in the highspeed image capturing mode. As shown in FIG. 5, raw data of 240 fpsreceived from the image sensor 101 are supplied to the recording device111 through the terminal r of the switch SW1 and the recording devicecontrol circuit 210. When a record request for an image that has beencaptured at a high speed is received from the control section 213according to the user's operation, raw data are recorded on therecording device 111.

The decimation section 221 of the conversion processing section 201decimates raw data by 1/4 such that the raw data of 60 fps are obtained.The raw data that have been decimated and size-converted are supplied tothe camera signal processing circuit 203 through the pre-processingcircuit 202. A signal for which the camera signal process has beenperformed in the camera signal processing circuit 203 is supplied to thedisplay section 112 through the display processing circuit 208 and animage that is being captured is displayed on the display section 112.

In the image capturing apparatus shown in FIG. 2, in the reproducingstate, the control section 213 controls the switches SW1 and SW2 of theconversion processing section 201 to select the terminal p side. FIG. 6shows signal flows in the reproducing state. As shown in FIG. 6, rawdata of the low screen rate (for example, 60 fps) are read from therecording device 111 according to the display performance of the displaysection 112 under the control of the recording device control circuit210. The raw data that have been read are supplied from the recordingdevice control circuit 210 to the pre-processing circuit 202 through theswitches SW1 and SW2 of the conversion processing section 201 and thesize adjustment section 222.

An output signal of the pre-processing circuit 202 is supplied to thedisplay section 112 through the camera signal processing circuit 203 andthe display processing circuit 208 and a reproduced image is displayedby the display section 112. For example, when only the screen rate hasbeen changed, the reproduced image becomes a slow motion reproducedimage, the time axis of which has been expanded four times than in therecording state. Instead, images may be captured by changing imagecapturing conditions (exposure condition and so forth) and when they arereproduced, the four types of captured images (any of still images ormoving images) may be compared. Instead, a signal that is read from therecording device 111 may be decimated so as to obtain a frame-by-framereproduction image.

Next, with reference to FIG. 7, an image capturing apparatus 300according to a second embodiment of the present invention will bedescribed. In the image capturing apparatus 300, a compression anddecompression circuit 301 is disposed between a conversion processingsection 201 and a recording device control circuit 210. The compressionand decompression circuit 301 performs a compression-encoding processaccording to an encoding system, for example, JPEG (Joint PhotographicExperts Group) or the like for raw data of the high screen rate receivedfrom the conversion processing section 201. Compression-encoded data arewritten to a recording device 111 under the control of the recordingdevice control circuit 210. As the compression-decompression encodingsystem, a binary data encoding system may be used instead of the JPEG.

FIG. 8 shows signal flows in the recording state. Raw data of the highscreen rate obtained in the high speed image capturing mode arecompression-encoded by the compression and decompression circuit 301 andthe compression-encoded data are recorded on a recording device 111through the recording device control circuit 210. Like the firstembodiment, an image signal whose screen rate has been converted intothe regular screen rate by the conversion processing section 201 andthat has been processed by the pre-processing circuit 202 and the camerasignal processing circuit 203 is displayed on the display section 112.

As shown in FIG. 9, in the reproducing state, encoded data of raw datathat have been read from the recording device 111 and supplied from therecording device control circuit 210 is expansion-decoded by thecompression and decompression circuit 301. Expansion-decoded raw dataare supplied from the conversion processing section 201 to the displaysection 112 through the pre-processing circuit 202, the camera signalprocessing circuit 203, and the display processing circuit 208 and areproduced image is displayed on the display section 112. The read speedof the recording device 111 is set up such that raw data of the regularscreen rate are read.

Next, with reference to FIG. 10, an image capturing apparatus 400according to a third embodiment of the present invention will bedescribed. In the image capturing apparatus 400, a simple pre-processingcircuit 401 and a simple camera signal processing circuit 402 aredisposed downstream of a signal of a conversion processing section 201so as to display a camera-through image that is being captured on thedisplay section 112. “Simple” means that these circuits generate animage that is displayed only on the display section 112 and the imagequality of an image to be displayed may be as low as that satisfying thepurpose of checking for an image of an object being captured. Forexample, the number of bits of a signal displayed on the display section112 is needed to be smaller than that of output data of an A/D converterof the pre-processing circuit 401. The simple pre-processing circuit 401may be omitted. With the simple structure, power consumption and heatgeneration for which an image is monitored in the high speed imagecapturing state can be reduced.

FIG. 11 shows signal flows in the recording state of the image capturingapparatus 400. In the high speed image capturing mode, raw data having ascreen rate of 240 fps received from the image sensor 101 are convertedinto raw data having the regular screen rate of 60 fps by the conversionprocessing section 201. Raw data having the regular screen rate aresupplied to the display processing circuit 208 through the simplepre-processing circuit 401, the simple camera signal processing circuit402, and a terminal r of a switch SW3 and the raw data are displayed bythe display section 112. In the high speed image capturing mode, when arecord command is issued, raw data having the high screen rate arerecorded on the recording device 111 through the recording devicecontrol circuit 210.

As shown in FIG. 12, in the reproducing state, raw data read from therecording device 111 are supplied from the recording device controlcircuit 210 to the pre-processing circuit 202, the raw data areprocessed by the camera signal processing circuit 203, and are sent tothe display section 112 through a terminal p of the switch SW3 and thedisplay processing circuit 208. The pre-processing circuit 202 and thecamera signal processing circuit 203 have the same structure as those ofthe foregoing first and second embodiments and a reproduced image havinga higher quality than an image being captured is displayed on thedisplay section 112.

Next, with reference to FIG. 13, an image capturing apparatus 500according to a forth embodiment of the present invention will bedescribed. The image capturing apparatus 500 has a simple pre-processingcircuit 401 and a simple camera signal processing circuit 402 disposeddownstream of a signal of a conversion processing section 201 so as todisplay a camera-through image being captured on the display section112. Thus, like the foregoing third embodiment, power consumption andheat generation that occur while an image is being monitored can bereduced. In addition, in a non-recording period, raw data can be readfrom a memory 502 at the regular screen rate and a camera signal processcan be performed for the raw data such that the resultant data arerecorded on a recording device 505.

In the image capturing apparatus 500, raw data received from theconversion processing section 201 are supplied to the memory 502 througha memory control circuit 501. The memory control circuit 501 controlswriting and reading image data to and from the memory 502. The memory502 is a FIFO (First In First Out) type memory that temporarily storesimage data received from the memory control circuit 501, for example anSDRAM (Synchronous Dynamic Random Access Memory) or the like. The memory502 performs buffering corresponding to throughputs of thepre-processing circuit 202 and the camera signal processing circuit 203.

An output signal of the camera signal processing circuit 203 is suppliedto a terminal r of a switch SW4. A terminal p of the switch SW4 and aterminal p of a switch SW3 are connected in common. The switch SW4 isconnected to a recording device control circuit 504. Connected to therecording device control circuit 504 is a recording device 505.

The recording device control circuit 504 controls writing and readingimage data to and from the recording device 505 through the switch SW4.Data stored in the recording device 505 are a luminance signal and acolor difference signal processed by the pre-processing circuit 202 andthe camera signal processing circuit 203. The recording device 505 maybe a magnetic tape, a semiconductor memory such as a flash memory, arecordable optical disc, a hard disk, or the like. The recording device505 is basically an attachable/detachable type. Instead, the recordingdevice 505 may not be attachable/detachable type and recorded data maybe output to the outside through a communication interface.

FIG. 14 shows signal flows in the recording state of the image capturingapparatus 500. In the high speed image capturing mode, raw data having ascreen rate of 240 fps received from the image sensor 101 is convertedinto raw data having the regular screen rate of 60 fps by the conversionprocessing section 201. The raw data having the regular screen rate issupplied to a display processing circuit 208 through the simplepre-processing circuit 401, the simple camera signal processing circuit402, and the terminal r of the switch SW3 and the raw data are displayedby the display section 112. When a record command is issued in the highspeed image capturing mode, the raw data having the high screen rate isrecorded on the memory 502 through the memory control circuit 501.

In a recording pause period in the high speed image capturing mode, forexample, in a recoding standby state where a hand is released from arecord button, raw data are read from the memory 502 and the raw dataare processed by the pre-processing circuit 202 and the camera signalprocessing circuit 203 and an output signal of the camera signalprocessing circuit 203 is recorded on the recording device 505 throughthe terminal r of the switch SW4 and the recording device controlcircuit 504. The raw data are read from the memory 502 at the regularscreen rate of 60 fps or a lower rate.

As shown in FIG. 15, in the reproducing state, raw data having theregular screen rate read from the recording device 505 are sent to thedisplay section 112 through the recording device control circuit 504,the terminal p of the switch SW4, and the display processing circuit208. A reproduced image reproduced from the recording device 505 isdisplayed by the display section 112. In the image capturing apparatus500, when the raw data are reproduced, the camera signal process is notrequired in comparison with that of the foregoing embodiment and therebywhen the raw data are reproduced, the power consumption can be reduced.

The present invention is not limited to the foregoing embodiments.Instead, various modifications of the embodiments can be performed basedon the spirit of the present invention. For example, data stored in therecording device 111 of the image capturing apparatus 400 (FIG. 10) ofthe third embodiment or data stored in the memory 502 or the recordingdevice 505 of the image capturing apparatus 500 (FIG. 13) of the fourthembodiment may be compressed.

In addition, the present invention can be applied to devices having animage capturing function such as a mobile phone and a PDA (PersonalDigital Assistants) as well as a camcorder and a digital still camera.In addition, the present invention can be applied to a processing deviceand a recording device for a captured image signal of a small camera fora television phone or a game software application connected to apersonal computer or the like.

1-70. (canceled)
 71. An image processing apparatus, comprising:circuitry configured to receive a first number of first frames of imagedata that have not yet been subjected to white balance processing, gammacorrection processing, and YC conversion processing at a first imagesize outputted by an image sensor that includes a color filter array;and at least one first storage medium encoded with executableinstructions that, when executed by the circuitry, cause the circuitryto: store the image data, without performing white balance processing,gamma correction processing, and YC conversion processing thereon, on asecond storage medium while the second storage medium is attached to theimage processing apparatus, the second storage medium being configuredto be detachable from the image processing apparatus so as to allow theimage data to be processed by an external device separate from the imageprocessing apparatus when the second storage medium is detached from theimage processing apparatus; and generate a first signal representing afirst white-balanced, gamma-corrected, and YC converted image sequencethat is to be outputted to the external device by processing the firstframes of image data to generate a second number of second frames ofimage data, and by performing at least one of white balance processing,gamma correction processing, or YC conversion processing on the secondframes of image data at a second image size which is lower than thefirst image size.
 72. The image processing apparatus of claim 71,wherein the at least one first storage medium is further encoded withexecutable instructions that, when executed by the circuitry, cause thecircuitry to: in a reproduction process, read the image data from thesecond storage medium; and generate a second signal representing asecond white-balanced, gamma-corrected, and YC converted image sequenceto be outputted to the second external device as a slow motionreproduced image by processing the image data.
 73. The image processingapparatus of claim 72, wherein the at least one first storage medium isfurther encoded with executable instructions that, when executed by thecircuitry, cause the circuitry to: in the reproduction process, generatethe second signal by processing the decompressed image data at a thirdframe rate which is lower than the first frame rate.
 74. The imageprocessing apparatus of claim 73, wherein the third frame rate is thesame as the second frame rate.
 75. The image processing apparatus ofclaim 71, wherein the at least one first storage medium is furtherencoded with executable instructions that, when executed by thecircuitry, cause the circuitry to: process the first and second framesof image data such that a ratio of the first number of frames to thesecond number of frames is the same as a ratio of the first frame rateto the second frame rate.
 76. The image processing apparatus of claim71, wherein the at least one first storage medium is further encodedwith executable instructions that, when executed by the circuitry, causethe circuitry to: generate the first signal such that the firstwhite-balanced, gamma-corrected, and YC converted image sequencecorresponds to at least some of the image data that is currently beingcaptured by the image sensor.
 77. The image processing apparatus ofclaim 75, wherein the at least one first storage medium is furtherencoded with executable instructions that, when executed by thecircuitry, cause the circuitry to: generate the second number of secondframes of image data by decimating the first number of first frames ofimage data.
 78. The image processing apparatus of claim 72, wherein: theimage sensor is further configured to output the first number of firstframes of image data throughout a first time period; and the at leastone first storage medium is further encoded with executable instructionsthat, when executed by the circuitry, cause the circuitry to, in thereproduction process, generate the second signal so that the secondwhite-balanced, gamma-corrected, and YC converted image sequence isoutputted to the second external device throughout a second time period,which is longer than the first time period.
 79. The image processingapparatus of claim 71, wherein the at least one first storage medium isfurther encoded with executable instructions that, when executed by thecircuitry, cause the circuitry to: convert the first number of firstframes of image data to the second number of second frames of image databy decimating the first number of first frames of image data.
 80. Theimage processing apparatus of claim 79, wherein the at least one firststorage medium is further encoded with executable instructions that,when executed by the circuitry, cause the circuitry to: convert thefirst number of first frames of image data to the second number ofsecond frames of image data by performing a size adjustment processingon at least some frames of image data.
 81. The image processingapparatus of claim 71, wherein the second frame rate is one fourth ofthe first frame rate.
 82. The image processing apparatus of claim 71,wherein the at least one first storage medium is further encoded withexecutable instructions that, when executed by the circuitry, cause thecircuitry to: generate the first signal by performing a simultaneousforming process on at least some frames of image data.
 83. The imageprocessing apparatus of claim 82, wherein the at least one first storagemedium is further encoded with executable instructions that, whenexecuted by the circuitry, cause the circuitry to: generate the firstsignal by performing each of white balance processing, gamma correctionprocessing, and YC conversion processing on the second frames of imagedata.
 84. The image processing apparatus of claim 71, wherein theexternal device comprises a liquid crystal display and the at least onefirst storage medium is further encoded with executable instructionsthat, when executed by the circuitry, cause the circuitry to: generatethe first signal by performing display processing on the second framesof image data, after performing the at least one of white balanceprocessing, gamma correction processing, and YC conversion processingthereon, and providing the first signal to the liquid crystal display soas to cause the liquid crystal display to display the firstwhite-balanced, gamma-corrected, and YC converted image sequence. 85.The image processing apparatus of claim 71, wherein the at least onefirst storage medium is further encoded with executable instructionsthat, when executed by the circuitry, control the circuitry such thatthe image data that is stored on the second storage medium is RAW imagedata.
 86. The image processing apparatus of claim 71, wherein the atleast one first storage medium is further encoded with executableinstructions that, when executed by the circuitry, cause the circuitryto: generate the first signal by performing white balance processing onthe second frames of image data at the second frame rate.
 87. The imageprocessing apparatus of claim 71, wherein the at least one first storagemedium is further encoded with executable instructions that, whenexecuted by the circuitry, cause the circuitry to: generate the firstsignal by performing gamma correction processing on the second frames ofimage data at the second frame rate.
 88. The image processing apparatusof claim 71, wherein the at least one first storage medium is furtherencoded with executable instructions that, when executed by thecircuitry, cause the circuitry to: generate the first signal byperforming YC conversion processing on the second frames of image dataat the second frame rate.
 89. The image processing apparatus of claim71, wherein the first frame rate is approximately 240 fps and the secondframe rate is approximately 60 fps.
 90. A method for operating an imageprocessing apparatus, comprising: receiving a first number of firstframes of image data that have not yet been subjected to white balanceprocessing, gamma correction processing, and YC conversion processing ata first image size outputted by an image sensor that includes a colorfilter array; storing the image data, without performing white balanceprocessing, gamma correction processing, and YC conversion processingthereon, on a second storage medium while the second storage medium isattached to the image processing apparatus, the second storage mediumbeing configured to be detachable from the image processing apparatus soas to allow the image data to be processed by an external deviceseparate from the image processing apparatus when the second storagemedium is detached from the image processing apparatus; and generating afirst signal representing a first white-balanced, gamma-corrected, andYC converted image sequence that is to be outputted to the externaldevice by processing the first frames of image data to generate a secondnumber of second frames of image data, and by performing at least one ofwhite balance processing, gamma correction processing, or YC conversionprocessing on the second frames of image data at a second image sizewhich is lower than the first image size.